<!DOCTYPE html>
<html xmlns:MadCap="http://www.madcapsoftware.com/Schemas/MadCap.xsd" class="_Skins_HTML5___Top_Navigation" lang="en-us" xml:lang="en-us" data-mc-search-type="Stem" data-mc-help-system-file-name="Default.xml" data-mc-path-to-help-system="../../" data-mc-has-content-body="True" data-mc-target-type="WebHelp2" data-mc-runtime-file-type="Topic;Default" data-mc-preload-images="false" data-mc-in-preview-mode="false" data-mc-toc-path="Evaluating Experiments with Inference Testing">
    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <meta charset="utf-8" />
        <meta http-equiv="X-UA-Compatible" content="IE=edge" />
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
        <link href="../../Skins/Default/Stylesheets/Slideshow.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/TextEffects.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Topic.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Components/Styles.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Components/Tablet.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Default/Stylesheets/Components/Mobile.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/foundation.6.2.3.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/Styles.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/Tablet.css" rel="stylesheet" type="text/css" data-mc-generated="True" />
        <link href="../../Skins/Fluid/Stylesheets/Mobile.css" rel="stylesheet" type="text/css" data-mc-generated="True" /><title>Batch Inference Example </title>
        <link href="../Resources/Stylesheets/Styles.css" rel="stylesheet" type="text/css" />
        <script src="../../Resources/Scripts/custom.modernizr.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/jquery.min.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/require.min.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/require.config.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/foundation.6.2.3_custom.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/plugins.min.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapGlobal.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapDom.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapUtilities.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapXhr.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapTextEffects.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapSlideshow.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapFeedback.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapDefault.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapHelpSystem.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.Breadcrumbs.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.MiniToc.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapToc.SideMenu.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapIndex.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapGlossary.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapParser.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapSearch.js" type="text/javascript">
        </script>
        <script src="../../Resources/Scripts/MadCapTopic.js" type="text/javascript">
        </script>
    </head>
    <body>
        <div class="foundation-wrap off-canvas-wrapper">
            <div class="off-canvas-wrapper-inner" data-off-canvas-wrapper="">
                <aside class="off-canvas position-left" id="offCanvas" data-off-canvas="" data-position="left" data-mc-ignore="true">
                    <ul class="off-canvas-drilldown vertical menu off-canvas-list" data-drilldown="" data-mc-back-link="Back" data-mc-css-tree-node-expanded="is-drilldown-submenu-parent" data-mc-css-tree-node-collapsed="is-drilldown-submenu-parent" data-mc-css-sub-menu="vertical menu slide-in-left is-drilldown-submenu" data-mc-include-indicator="False" data-mc-include-icon="False" data-mc-include-parent-link="True" data-mc-include-back="True" data-mc-defer-expand-event="True" data-mc-expand-event="click.zf.drilldown" data-mc-toc="True">
                    </ul>
                </aside>
                <div class="off-canvas-content inner-wrap" data-off-canvas-content="">
                    <div data-sticky-container="" class="title-bar-container">
                        <nav class="title-bar tab-bar sticky" data-sticky="" data-options="marginTop:0" style="width:100%" data-sticky-on="only screen and (max-width: 1000px)" data-mc-ignore="true">
                            <div class="middle title-bar-section outer-row clearfix">
                                <div class="menu-icon-container relative clearfix">
                                    <button class="menu-icon" data-toggle="offCanvas"><span></span>
                                    </button>
                                </div>
                            </div>
                            <div class="title-bar-layout outer-row">
                                <div class="logo-wrapper"><a class="logo" href="index.htm" alt="Logo"></a>
                                </div>
                                <div class="navigation-wrapper nocontent">
                                    <ul class="navigation clearfix" data-mc-css-tree-node-has-children="has-children" data-mc-css-sub-menu="sub-menu" data-mc-expand-event="mouseenter" data-mc-top-nav-menu="True" data-mc-max-depth="3" data-mc-include-icon="False" data-mc-include-indicator="False" data-mc-include-children="True" data-mc-include-siblings="True" data-mc-include-parent="True" data-mc-toc="True">
                                        <li class="placeholder" style="visibility:hidden"><a>placeholder</a>
                                        </li>
                                    </ul>
                                </div>
                                <div class="nav-search-wrapper">
                                    <div class="nav-search row">
                                        <form class="search" action="#">
                                            <div class="search-bar search-bar-container needs-pie">
                                                <input class="search-field needs-pie" type="search" placeholder="Search" />
                                                <div class="search-filter-wrapper">
                                                    <div class="search-filter">
                                                        <div class="search-filter-content">
                                                            <ul>
                                                                <li>All Files</li>
                                                            </ul>
                                                        </div>
                                                    </div>
                                                </div>
                                                <div class="search-submit-wrapper" dir="ltr">
                                                    <div class="search-submit" title="Search">
                                                    </div>
                                                </div>
                                            </div>
                                        </form>
                                    </div>
                                </div>
                            </div>
                        </nav>
                    </div>
                    <section class="main-section">
                        <div class="row outer-row sidenav-layout">
                            <div class="sidenav-wrapper">
                                <div class="sidenav-container">
                                    <ul class="off-canvas-accordion vertical menu sidenav" data-accordion-menu="" data-mc-css-tree-node-expanded="is-accordion-submenu-parent" data-mc-css-tree-node-collapsed="is-accordion-submenu-parent" data-mc-css-sub-menu="vertical menu accordion-menu is-accordion-submenu nested" data-mc-include-indicator="False" data-mc-include-icon="False" data-mc-include-parent-link="False" data-mc-include-back="False" data-mc-defer-expand-event="True" data-mc-expand-event="click.zf.accordionMenu" data-mc-toc="True" data-mc-side-nav-menu="True">
                                    </ul>
                                </div>
                            </div>
                            <div class="body-container" data-mc-content-body="True">
                                <h1>Batch Inference Example </h1>
                                <h2>Example Flow</h2>
                                <p>Following is the general flow of this example:&#160;</p>
                                <ol>
                                    <li value="1">The user has acquired the dataset and the trained model.</li>
                                    <li value="2">The user converts dataset into serialized protocol buffers (PBs). Refer to <a href="getting_started.htm">https://developers.google.com/protocol-buffers/</a> as one information source.</li>
                                    <li value="3">The user invokes <span style="font-family: 'Courier New';">nctl mount</span>.</li>
                                    <li value="4">The user mounts the Samba shared folder by invoking the command displayed in the previous step.</li>
                                    <li value="5">The user copies the serialized PBs and the trained model to the just-mounted share.</li>
                                    <li value="6">The user runs <span style="font-family: 'Courier New';">nctl predict batch</span> command.</li>
                                </ol>
                                <h2>MNIST Example</h2>
                                <p>You need to have preprocessed MNIST data for feeding the batch inference. You can use already-preprocessed data located in 'parsed' directory, or generate data on your own (refer to <a href="#MNIST" class="MCXref xref" xrefformat="{paratext}">MNIST Data Preprocessing</a>).</p>
                                <ol>
                                    <li value="1">Mount Samba input share to your directory, assumed /mnt/input . Use the command printed by <span style="font-family: 'Courier New';">nctl mount</span>.</li>
                                    <li value="2">Copy 'parsed' and 'trained_mnist_model' to <span style="font-family: 'Courier New';">/mnt/input</span>.<br /></li>
                                    <li value="3">Enter the following command:&#160;<br /><span style="font-family: 'Courier New';">$ nctl predict batch --model-location /mnt/input/home/trained_mnist_model --data /mnt/input/home/parsed --model-name mnist</span><br /></li>
                                </ol>
                                <p><b>Notes</b>:</p>
                                <ul>
                                    <li value="1">Paths provided in locations such as <span style="font-family: 'Courier New';">--model-location</span> and <span style="font-family: 'Courier New';">--data</span> need to point for files/directory from the container's context, not from user's filesystem or mounts. These paths can be mapped using instructions from <span style="font-family: 'Courier New';">nctl mount</span>. For example, if you've mounted Samba <span style="font-family: 'Courier New';">/input</span> and copied files there, you should pass <span style="font-family: 'Courier New';">/mnt/input/home/&lt;file&gt;</span> .</li>
                                    <li value="2"><span style="font-family: 'Courier New';">--model-name</span> is optional, but it must match the model name provided during data preprocessing, since generated requests must provide which servable they target. In the <span style="font-family: 'Courier New';">mnist_converter_pb.py</span> script you can find <span style="font-family: 'Courier New';">request.model_spec.name = 'mnist'</span>, which saves model name in requests, and that name must match value passed as <span style="font-family: 'Courier New';">--model-name</span>. If not provided it assumes that model name is equal to last directory in model location: <span style="font-family: 'Courier New';">/mnt/input/home/trained_mnist_model -&gt; trained_mnist_model</span></li>
                                </ul>
                                <h2><a name="MNIST"></a>MNIST Data Preprocessing</h2>
                                <p>Perform the following steps:&#160;</p>
                                <ol>
                                    <li value="1">Create venv. Enter the following command:<br /><span style="font-family: 'Courier New';">$ make venv</span></li>
                                    <li value="2">Run the <span style="font-family: 'Courier New';">mnist_converter_pb.py</span> script using just generated venv:<br /></li>
                                </ol>
                                <blockquote>
                                    <p style="font-family: 'Courier New';">$ source .venv/bin/activate<br />$ python mnist_converter_pb.py</p>
                                </blockquote>
                                <h2>Description of Files</h2>
                                <ul>
                                    <li value="1">output - example PBs responses from batch inference downloaded after nctl predict batch process.</li>
                                    <li value="2">parsed - example PBs parsed MNIST dataset generated by mnist_converter_pb.py script ready as an input for nctl predict batch.</li>
                                    <li value="3">trained_mnist_model - example MNIST trained model used for inference.</li>
                                    <li value="4">batch_client.py - prototype of batch inference wrapper client.</li>
                                    <li value="5">checker.py - script checking MNIST inference error rate from PBs responses.</li>
                                    <li value="6">mnist_converter_pb.py - example preprocessing script for preparing predict batch input. handles downloading MNIST dataset and saving PBs to 'parsed' folder.</li>
                                    <li value="7">mnist_input_data.py - dependency for mnist_converter_pb.py script for managing original MNIST dataset.</li>
                                    <li value="8">requirements.txt - required Python dependencies for invoking all Python scripts here.</li>
                                    <li value="9">requirements-dev.txt - optional Python dependency for e.g. auto formatting Python code.</li>
                                </ul>
                                <h2>References</h2>
                                <ul>
                                    <li value="1"><a href="template_packs.htm">https://www.tensorflow.org/serving/serving_basic</a>
                                    </li>
                                    <li value="2"><a href="template_packs.htm">https://developers.google.com/protocol-buffers/docs/pythontutorial</a>
                                    </li>
                                    <li value="3"><a href="template_packs.htm">https://github.com/tensorflow/serving/blob/master/tensorflow_serving/example/mnist_client.py</a>
                                    </li>
                                    <li value="4"><a href="template_packs.htm">https://www.tensorflow.org/serving/docker</a>
                                    </li>
                                </ul>
                            </div>
                        </div>
                    </section><a data-close="true"></a>
                </div>
            </div>
            <script>/* <![CDATA[ */$(document).foundation();/* ]]> */</script>
        </div>
    </body>
</html>